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Abstract 

We consider the convex feasibility problem (CFP) in Hilbert space 
and concentrate on the study of string-averaging projection (SAP) 
methods for the CFP, analyzing their convergence and their pertur- 
bation resilience. In the past, SAP methods were formulated with a 
single predetermined set of strings and a single predetermined set of 
weights. Here we extend the scope of the family of SAP methods to 
allow iteration-index-dependent variable strings and weights and term 
such methods dynamic string-averaging projection (DSAP) methods. 
The bounded perturbation resilience of DSAP methods is relevant 
and important for their possible use in the framework of the recently 
developed superiorization heuristic methodology for constrained min- 
imization problems. 
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1 Introduction 

In this paper we consider the convex feasibility problem (CFP) in Hilbert 
space H. Let C%, C2, . . . , C m be nonempty closed convex subsets of H, where 
m is a natural number, and define 

C := atxC,- (1) 

Assuming consistency, i.e., that C 7^ 0, the CFP requires to find an el- 
ement x* G C. We concentrate on the study of string- averaging projection 
(SAP) methods for the CFP and analyze their convergence and their pertur- 
bation resilience. SAP methods were first introduced in [11] and subsequently 
studied further in [U El [121 13 US EH] , see also Example 5.20]. They were 
also employed in applications [271 I2H] • 

The class of projection methods is understood here as the class of meth- 
ods that have the property that they can reach an aim related to the family 
of sets {Ci, C2, . . . , C m }, such as, but not only, solving the CFP, or solv- 
ing an optimization problem with these sets as constraints, by performing 
projections (orthogonal, i.e., least Euclidean distance, or others) onto the in- 
dividual sets Ci. The advantage of such methods occurs in situations where 
projections onto the individual sets are computationally simple to perform. 
Such methods have been in recent decades extensively investigated math- 
ematically and used experimentally with great success on some huge and 
sparse real-world applications, consult, e.g., [H [HI [T71 [18] and the books 

[SIEIEIIISKISIEIIEIES]. 

Within the class of projection methods, SAP methods do not represent 
a single algorithm but rather, what might be called, an algorithmic scheme, 
which means that by making a specific choice of strings and weights in SAP, 
along with choices of other parameters in the scheme, a deterministic algo- 
rithm for the problem at hand can be obtained. 
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In all these works, SAP methods were formulated with a single predeter- 
mined set of strings and a single predetermined set of weights. Here we extend 
the scope of the family of SAP methods to allow iteration-index-dependent 
variable strings and weights. We term such SAP methods dynamic string- 
averaging projection (DSAP) methods. This is reminiscence of the analogous 
development of block-iterative projection (BIP) methods for the CFP wherein 
iteration- index-dependent variable blocks and weights are permitted [T] . For 
such DSAP methods we prove here convergence and bounded perturbation 
resilience. 

The significance of DSAP in practice cannot be exaggerated. SAP meth- 
ods, in their earlier non-dynamic versions, have been applied to the important 
real- world application of proton Computerized Tomography (pCT), see, e.g., 
[28l|2Z], which presents a computationally huge-size problem. The efforts to 
use parallel computing for the application of SAP to pCT is ongoing and will 
benefit from the DSAP. This is so because the flexibility of varying string 
lengths, string members and weights dynamically has direct bearing on load 
balancing between processors that run in parallel and should be loaded in a 
way that will minimize idle time of processors that await others to complete 
their jobs. Such experimental work will hopefully see light elsewhere. 

The so extended DSAP algorithmic scheme is presented in Section [2] and 
the convergence analysis of it is done in Section |3j In Section @] we quote the 
definition of bounded perturbation resilience and prove that the DSAP with 
iteration-index-dependent variable strings and weights is bounded perturba- 
tion resilient. There we also comment about the importance and relevance 
of this bounded perturbation resilience to the recently developed superior- 
ization heuristic methodology. 

2 The string-averaging projection method and 
the dynamic SAP with variable strings and 
weights 

Let if be a real Hilbert space with inner product (•, •) and induced norm 
|| • ||. Originally, string-averaging is more general than SAP because it can 
employ operators other than projections and convex combinations. But, on 
the other hand, it is formulated for fixed strings as follows. Let the string I t 
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be an ordered subset of {1,2, ... ,m} of the form 

h = (ii,«2> • • • ;4i(t))> ( 2 ) 

with m(t) the number of elements in I t , for t — 1,2, ... , M, where M is the 
number of strings. Suppose that there is a set S C such that there are 
operators R\, R2, ■ ■ ■ , R m mapping S into 5 and an additional operator R 
which maps S M into S. 

Algorithm 1 The string-averaging algorithmic scheme of /J JJ/ 
Initialization: x° G 5 is arbitrary. 
Iterative Step: given the current iterate x k , 
(i) calculate, for all t = 1,2,..., M, 

T t (x k ) = R U) ---R lt R l{ (x k ), (3) 

(ii) and then calculate 

x k+l = R{T x {x k ),T 2 {x k ), . . .,T M (x k )). (4) 

For every t = 1,2,..., M, this algorithmic scheme applies to x k succes- 
sively the operators whose indices belong to the t-th string. This can be done 
in parallel for all strings and then the operator R maps all end-points onto 
the next iterate x k+1 . This is indeed an algorithm provided that the opera- 
tors {Ri}™ =1 and R all have algorithmic implementations. In this framework 
we get a sequential algorithm by the choice M = 1 and I\ — (1, 2, . . . , m) and 
a simultaneous algorithm by the choice M = m and I t = (t), t = 1, 2, . . . , M. 

Now we proceed to construct our proposed DSAP method with variable 
strings and weights. For each x G H, nonempty set E C H and r > define 
the distance 

d{x,E) = mi{\\x -y\\\ y e E} (5) 

and the closed ball 

B(x,r) = {y G H \ \\x — y\\ < r}. (6) 
The following proposition and corollary are well-known. 
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Proposition 2 If D be a nonempty closed convex subset of H then for each 
x G H there is a unique point Pd(x) G D, called the projection of x 
onto D, satisfying 

\\x - P D {x)\\ = M{\\x - y\\ | yED}. (7) 

Moreover, 

\\P D (x) - P D (y)\\ <\\x-y\\ for all x, ye H, (8) 
and for each x G H and each z G D, 

(z-P D (x),x-P D (x)) < 0. (9) 

Corollary 3 If D be a nonempty closed convex subset of H then for each 
x G H and each z G D, 

\\z-P D (x)\\ 2 + \\x-P D (x)\\ 2 < \\z-x\\ 2 . (10) 

For i = 1,2,..., m, we denote Pi = Pq v An index vector is a vector 
t = (ti,t 2 , ■ ■ ■ ,t p ) such that U G {1, 2, ... , to} for alH = 1, 2, . . . ,p. For a 
given index vector t = (ti, ■ ■ ■ , t q ) we denote its length by pit) = q, and 
define the operator P[t] as the product of the individual projections onto the 
sets whose indices appear in the index vector t, namely, 

P[t]:=P tq P k _ 1 ---P tl , (11) 

and call it a string operator. A finite set Q of index vectors is called fit if for 
each i e {1,2,..., to}, there exists a vector t = (t 1: t 2 , . . . , t p ) G f2 such that 
t s — i for some s G {1, 2, . . . ,p}. For each index vector t the string operator 
is nonexpansive, since the individual projections are, i.e., 

\\P[t](x) - P[t](y)\\ < \\x-y\\ ioiaRx,yeH, (12) 

and also 

P[t](x) = x for all x G C. (13) 

Denote by Ai the collection of all pairs (fl,w), where Q is a fit finite set 
of index vectors and 

w : ->■ (0, oo) is such that w(t) = 1. (14) 

ten 
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A pair (Q, w) G Ai and the function w were called in [5] an amalgamator 
and a fit weight function, respectively. For any (Q, w) G M. define the convex 
combination of the end-points of all strings defined by members of Q by 

Pn,w(x) ■= ^2w{t)P[t](x), xEH. (15) 

ten 

It is easy to see that 

\\Pn,w(x) - Pn,w(y)\\ < \\x -y\\ for all x,y G H, (16) 

and that 

Pq,w( x ) — x f° r ah x E C. (17) 

We will make use of the following bounded regularity condition, see [21 Def- 
inition 5.1]. 

Condition 4 For each e > and each M > there exists 5 = 5(e, M) > 
such that for each x G 5(0, M) satisfying d(x, Ci) < 5, i = 1,2, ... ,m, the 
inequality d(x, C) < e holds. 

The next proposition follows from [2J Proposition 5.4(iii)] but we present 
its proof here for the reader's convenience. 

Proposition 5 If the space H is finite- dimensional then Condition^ holds. 

Proof. Assume to the contrary that there exist e > 0, M > and a sequence 
{x k }%L C .8(0, M) such that 

for each integer k > 1, d(x k , Ci) < 1/k, i — 1, 2, . . . , m and d(x h , C) > e. 

(18) 

We assume, without loss of generality, that there exists the limit 

lim x k = x. (19) 

Then the closedness of B(Q, M) and ( ITS]) imply that 

x G 5(0, M) nC h i = 1,2,..., m. (20) 

Together with (JTJ) and ( 119]) this implies that rf(x fc , C) < e/2 for all sufficiently 
large natural numbers k, contradicting (JTSJ and proving the proposition. ■ 



6 



In the sequel we will assume that Condition H] holds. We fix a number 
A G (0, 1/m) and an integer q > m and denote by M.^ = _M*(A, q) the set 
of all (fi, w) e M. such that the lengths of the strings are bounded and the 
weights are all bounded away from zero, namely, 

M* := {(tt,w) eM \ p(t) < q and w{t) > A for all t G Q}. (21) 

The dynamic string-averaging projection (DSAP) method with variable 
strings and variable weights is described by the following algorithm. 

Algorithm 6 The DSAP method with variable strings and vari- 
able weights 

Initialization: select an arbitrary x° G H , 

Iterative step: given a current iteration vector x k pick a pair (Q^jWh) G 
M.* and calculate the next iteration vector by 

x k+1 = Pn k , Wk {x k ). (22) 

3 Convergence analysis 

In this section we present our convergence analysis for the DSAP method 
with variable strings and variable weights, Algorithm O The main theorem 
is the following. 

Theorem 7 Let the following assumptions hold: 

(i) M > is such that 

B(0, M ) n C ^ 0. (23) 

(ii) e > 0, M > and 5 > are such that 

ifx G B(0,2M + M) and d(x,C t ) < 5, i = 1,2, . . .,m, then d(x,C) < e/4. 

(24) 

(Hi) 7 is a positive number that satisfies 

qi 1/2 < S. (25) 
(iv) ko is a natural number that satisfies 

k > ( 7 A)- 1 (M + M ) 2 . (26) 
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Under these assumptions, if Condition^ holds then any sequence {x k }^L C 
H, generated by Algorithm^ with < M, converges in the norm of H, 
limfc_>oo x k G C and 

\\x k — lim x s \ \ < e for all integers k > k . (27) 

We use the notation and the definitions from Section [2] and prove first 
the next two lemmas as tools for the proof of Theorem [7J 

Lemma 8 Let t = (ti, tz, ■ ■ ■ , t p ) be an index vector, x G H and z e C. 
Then 

\\z-x\\ 2 > \\z - P[t](x)\\ 2 + \\x - P tl (x)\\ 2 (28) 

V 

+ \\Pt i+ iPt i ---Pt,{x)-P ti ---PtM\\ 2 - (29) 

% = 1 
i < p 

Proof. By Corollary [3j 

\\z-x\\ 2 > \\z - P tl (x)\\ 2 + \\x - P tl (x)\\ 2 (30) 

and using the same corollary we also have, for each integer i satisfying 1 < 

i < p, 

\\z-P u ---P tl x\\ 2 >\\z-P ti+1 P ti ---P tl (x)\\ 2 

+ \\P ti+l P ti ...P tl x-P ti ---P tl (x)\\ 2 . (31) 

Combining fl3U|) and f|3T]) we obtain (|28|) and the lemma is proved. ■ 

For an index vector t = (ti,t 2 , . . . , t p (t)) and a vector x G H let us define 
the function 

v 

^t](x):=\\x-P tl (x)\\ 2 + \\Pt i+ xP ti ---Pt^)-Pt i ---Pt 1 (x)\\ 2 . (32) 

i = 1 

i < p 

Lemma |8]and ( l32l) then imply that, for each x G H and each zeC, 

\\z-x\\ 2 > \\z-P[t}(x)\\ 2 + (f)[t}(x). (33) 
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Lemma 9 Let x G H, z G C and (Q,w) G M.*. Then 

\\z-x\\ 2 > \\z - Pn, w {x)\\ 2 + A^0[t](x). (34) 

ten 

Proof. Since the function u — > \ \u — z\ \ 2 , u G H, is convex it follows from 
flUD, (USD; (EHD , and the definition of .M* in fl2I]) that 

||z-P^(x)|| 2 =|| 2 -^ W (t)P[t](x)|| 2 <^ W (t)||,-P[t](x)|| 2 

ten ten 

< ^2w(t) {\\z - x\\ 2 - <p[t](x)) = \\z - x\\ 2 -^2w(t)(j)[t](x) 

ten ten 

< \\z-x\\ 2 - A^2<f)[t](x). (35) 

ten 

This implies ( 13~4"|) and completes the proof. ■ 
Now we are ready to prove Theorem [7J 

Proof of Theorem We first wish to show that there is a natural 
number £ < k such that 

5>[f](z<- 1 )<7- (36) 

To this end we assume to the contrary, that 

for all k = 1,2,..., k , ^ 4>[t\{x h ~ l ) > 7, (37) 

ten fc 

and take some 9 G -E>(0, M )C\C. By LemmaEl by the fact that (f2&, u^) G Ai* 
for all fc > 0, by ([22]) and by (J37J, we have 

||0 -x^ 1 ! | 2 > \\6 - x k \\ 2 + AJ2 ^K^ 1 ) > \\9-x k \\ 2 + A 7 . (38) 

ten k 

By the choice of 9 and the fact that < M we obtain 

(M + M ) 2 > ||0-s°|| 2 - ||#-a; fco || 2 
fco 

= Y^\\9 - x^W 2 - \\9- x k \\ 2 ) > k Aj (39) 
fc=i 
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which implies 

fc <(A 7 )- 1 (M + M ) 2 . (40) 

This contradicts Assumption (iv) of the theorem thus showing that there 
exists a natural number i < k such that (|36|) holds. 

From LemmalHl the choice of 9, the fact that (Qk, Wk) G -M* for all k > 0, 
the iterative step (152]) and < M, we conclude that 

\\9-x l ~ l \\ 2 < \\9-x°\\ 2 < (M + M) 2 , (41) 

which yields by the triangle inequality 

\\x e ~ 1 \\<2M + M. (42) 

In order to use the last inequality to invoke Assumption (ii) of the theorem 
we show next that 

d(x e -\Ci) < 5, i = 1,2,..., m. (43) 

Assume that s G {1,2, ...,m}. Since the set is fit there is a t = 
(ti, t 2 , ■ ■ ■ , t p {t)) £ such that 

s = t q for some qe {1,2,.. .,p(t)}. (44) 

From (13T)]) and the fact that t G Qg, we know that </>[i](a;^ _1 ) < 7. Together 
with ( 132]) this implies that 

||^- 1 _p ti( ^-l ) ||< 7 l/2 ) (45) 

therefore, for any index 1 < i < p(t) satisfying i < p(t), 

\\P ti+1 P ti ■ ■ ■ PtA^- 1 ) -P ti --- PtA^W < 7 1/2 - (46) 

The fact that (Qk,Wk) G .M* for all k > guarantees that p{t) < q, and 
the S whose existence is guaranteed by Condition H] Assumption (iii) of the 
theorem, along with (T43]) and ( 145]) imply that for any i G {1, . . . ,p(t)}, 

\\P U ■ ■■P tl (x e - 1 ) - x^W < i^' 2 < p(t)^ 2 < qj 1 ' 2 < 5 (47) 

and thus that 

d(x e ~\C ti ) <5. (48) 
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Together with (j44p this implies that 

d{x t - 1 ,C s )<5. (49) 

Since (l4"9"j) holds for any s G {1,2,..., m}, we use (T4"2"j) and Assumption (ii) 
of the theorem to state that 

d{x l -\C) < e/4 (50) 

and that there is a z G H such that 

^ G C and ll^ -1 - z|| < e/3. (51) 

By (USD, (HZD> (EU), the fact that (fl fc ,«) fc )6M* for all fc > and the iterative 
step (12"2"1) we have 

lk fc - z\\ < \ \x e ~ l — z\\ < e/3 for all integers k > £ — 1. (52) 

This implies that for all integers k\, ft2 > — 1 it is true that | — a; fc2 1| < e. 
Since e > is arbitrary it follows that {x k }^L is a Cauchy sequence and 
that the limit lim^oo x k in the norm exists. By (|52|) 

II*- lim x k \ \ < e/3. (53) 

Since e > is arbitrary it follows from (I5TI) that lim fe _ s . 00 x fc G C. By (152"]) . 
since £ < ho, and using (13 7p for all integers k > k we may write 

||a; fc - lim x s || < | |x fc - z| | + | |z - lim x s \ \ < e/3 + e/3 (54) 
which completes the proof of Theorem [71 ■ 

4 Perturbation resilience of dynamic string- 
averaging with variable strings and weights 

In this section we prove the bounded perturbation resilience of the DSAP 
method with variable strings and weights. We use the notations and the 
definitions from the previous sections. The next definition was originally 
given with a finite-dimensional Euclidean space R J instead of the Hilbert 
space H that we inserted into it below. 
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Definition 10 fTOj Definition 1] Given a problem T, an algorithmic oper- 
ator A : H — >■ H is said to be bounded perturbations resilient if the 

following is true: if the sequence {x k }^L , generated by x k+1 = A(x k ), for 
all k > 0, converges to a solution of T for all x° G H , then any sequence 
{y k }T=o of points in H generated by y k+1 = A(y k + (3kV k ), for all k > 0, also 
converges to a solution of T provided that, for all k > 0, (5kV k are bounded 

oo 

perturbations, meaning that (3k > for all k > such that 2^0k < oo 

and the sequence {v k } k x L is bounded. 

We will make use of the following theorem that was proved in [61 Theorem 
3.2]. 

Theorem 11 Let (Y, p) be a complete metric space, let F C Y be a nonempty 
closed set, and let % : Y — > Y , i — 1,2, ... , satisfy 

p(Ti(x),Ti(y)) < p(x,y) for all x,y <EY and all integers i > 1, (55) 

and 

Ti(z) = z for each z e F and each integer % > 1. (56) 

Assume that for each x G Y and integer q > 1, the sequence {T n ■ ■ ■ T q (x)}%L q 
converges to an element of F. Let x° G Y, { r y n }'^ =1 C (0, oo), Y^Liln < °°, 
{x n }'^ =Q C Y , and suppose that for all integers n > 0, 

p(x n+1 ,T n+1 {x n )) < 7n+1 . (57) 

Then the sequence {x n }^ =0 converges to an element of F. 

The next theorem establishes the bounded perturbations resilience of 
DSAP. 

Theorem 12 Let Ci, C2, ■ ■ ■ , C m be nonempty closed convex subsets of H, 
where m is a natural number, C := H^Cj 7^ 0, let {f3k} k x L be a sequence 
of nonnegative numbers such that Y^k=oh < 00, let {v k }^L C H be a 
norm bounded sequence, let {(Qk,Wk)}kLi C for all k > 0, and let 

x° G H. Then any sequence {xk}^L , generated by Algorithm&in which |H| ) 
is replaced by 

x k+1 = Pn k , Wk (x k + (3 k v k ), (58) 
converges in the norm of H and its limit belongs to C. 
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Proof. The proof follows from Theorem [7] and from Theorem [TTJ ■ 

We conclude with a comment about the importance and relevance of this 
bounded perturbation resilience to the recently developed superiorization 
methodology. The superiorization methodology was first proposed (although 
without using this term) in [5] and subsequently investigated and developed 
further in [TTJ], EDI EH ESI ESI EB]- For the case of a minimization problem 
of an objective function <p over a family of constraints {Cj}^, where each 
set Ci is a nonempty closed convex subset of R n it works as follows. It 
applies to C = V^ =x Ci a feasibility- seeking projection method capable of using 
projections onto the individual sets C* in order to generate a sequence {x k }^L 
that converges to a point x* G C. It applies to C only such a feasibility- 
seeking projection method which is bounded perturbation resilient. Doing 
so, the superiorization method exploits this perturbation resilience to perform 
objective function reduction steps by doing negative subgradient moves with 
certain step sizes. Thus, in superiorization the feasibility-seeking algorithm 
leads the overall process and uses permissible perturbations, that do not spoil 
the feasibility-seeking, to periodically jump out from the overall process to 
do the subgradient objective function reduction step. 

This has a great potential computational advantage and poses also in- 
teresting mathematical questions. It has been shown to be advantageous 
in some real-world problems in image reconstruction from projections, see 
the above mentioned references. Theorem [TJ] here, which established the 
bounded perturbations resilience of DSAP methods, makes it now possible 
to use, when the need arises, also DSAP methods within the framework of 
the superiorization heuristic methodology. 
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